Undergraduate Research Methods in Psychology
Department of Psychology
However, we have designs that can look at two (or more) IVs at once and see their individual and combined impact on the DV!
We refer to these as factorial designs, and they can be helpful in unpacking nuance in certain relationships.
We can add a second (and third) independent variable if we are curious about more than one.
In addition to the individual effects of both of the IVs, we also get an interaction effect that describes how they change each other’s relationship with the outcome.
We can see this even in our personal experiences, and many relationships do depend on other factors
Example: I am assessing how spicy I like my food (on a scale of 1 to 10; my outcome). First, is it cold or hot outside (IV 1)? Second, am I eating Thai or Italian (IV 2)? It is possible that my answer will be different based upon both of the IVs.
4 Possible Outcomes:
I like all of my food spicier when it is hot - Weather effect, but not food
Whether I like by food spicy or not depends on both the weather, and type of food - interaction effect
Specifically, we are looking to see whether we have a crossover interaction, like in the graph below:
When we work with more than one IV, we use a factorial design.
This creates more outcome unique conditions = # of Conditions in IV 1 x # of Conditions in IV 2 = total number of conditions
Both IVs do not have to be manipulated. Often, one will be some categorical, measured trait (e.g., gender, ethnicity, etc.)
In addition to our statistics, we should show these differences in plots! Interaction effects become especially clear with visual evidence.
Factorial designs can help us find whether outcomes are different for different types of people, or maybe an intervention only works if another intervention is present
A strong intervention may not be as effective in a different group of people.
This can be a boon to our external validity, as we demonstrate findings in a more heterogeneous group.
We also can establish whether one variable appears to moderate another on the relationship with the outcome variable.
For some theoretical reasons, we may have good reason to believe that an effect differs based on some demographic variable.
Example: I have a new intervention meant to encourage flexibility in learning and taking in new content. However, I recognize that the neuroplasticity of older adults is just lesser in general. Therefore, I believe my intervention will likely be more effective for younger adults, than it will for older adults.
In essence, we may be able to add nuance and “it depends” to our hypotheses and investigate with factorial designs.
Just like with other experiments, we can lay out a factorial design as being between-groups or within-groups.
But, we can designate each variable as between or within, leading to a total of 3 possible designs:
This is when all IVs are between-groups (i.e., participants are arranged into entirely separate groups)
One nuance is that this will likely require the largest sample size, as each group will have about 1/4th the total number of participants
Much like with previous within-groups designs, this is when participants see every possible condition.
One thing to watch out for is the need for counterbalancing to prevent order effects
Example: I am interested in seeing whether a certain note-taking strategy and a review strategy help performance on a test - so I have the same people counterbalances to different combinations of both conditions.
This is when one IV is between-groups, and the other is within-group.
This is fairly common if we have one demographic variable (between-groups) and one manipulated variable that both demographics are exposed to each level (within-groups).
Example: I am determining whether first-generation college students or legacy students benefit more from a mentoring program, in terms of confidence. Both first-gen and legacy students have a period that they are exposed to the mentoring program and a time period without.
Prof. Paul Moes: “God himself cannot interpret a 4-way interaction - neither can you”
We can do 3 IVs, but with each additional variable the interpretation becomes exponentially more difficult and complicated.
Remember to think carefully about what sorts of conclusions you can draw with a design before you use it, and whether an alternative provides a more simpler conclusion.
Look for words like …
You may also see phrasing like “2 x 2 design”, referring to the two conditions of each IV.
Sometimes a multiple regression model and a factorial design can be described in somewhat similar conclusions - you might have to work through the results to figure out which one was used
Review of Discussion and MC Questions
Remember to do the Q & A / Lecture Check-in!
Week 12 Lecture - Complex Experiments || Undergraduate Research Methods in Psychology